Home | | Latest | About | Random
# 46 Orthogonal diagonalization and real spectral theorem.
Let us now turn our attention to a special class of matrices: Real symmetric matrices. These matrices show up often enough in application, and as it turns out they have very nice properties:
> **Theorem. Real spectral theorem.**
> Let $A$ be a real symmetric matrix. Then $A$ is orthogonally diagonalizable. That is, there exists an **orthonormal eigenbasis**, and that we can factorize the matrix $A$ as $$
A = Q D Q^{T}
$$ where $Q$ is an orthogonal matrix, and $D$ diagonal. Furthermore, if $A$ is orthogonally diagonalizable, then $A$ is real symmetric. This fact is if and only if, and a characterization of real symmetric matrices.
**Remark.** Recall that an orthogonal matrix $Q$ is a square matrix with orthonormal column, and its inverse is its transpose, $Q^{-1}=Q^{T}$.
To prove the forward direction of real spectral theorem, the existence of such orthogonal diagonalization, one typically would need induction, which isn't difficult but we will skip it. The converse however is easy to show.
$(\impliedby)$ Suppose $A$ is orthogonally diagonalizable, that $A = QDQ^{T}$ for some orthogonal matrix $Q$ and diagonal matrix $D$. We will show that $A$ is symmetric. Indeed, we compute $$
A^{T} = (QDQ^{T})^{T} = (Q^{T})^{T}D^{T}Q^{T} = QDQ^{T}= A,
$$so $A^{T} = A$, that is $A$ is symmetric. $\blacksquare$
Additionally, we will show that eigenvectors of different eigenvalues of a real symmetric matrix are orthogonal to each other:
> **Lemma.**
> Let $A$ be a real symmetric matrix. Suppose $v$ is an eigenvector of $A$ with eigenvalue $\lambda$, and suppose $w$ is an eigenvector of $A$ with eigenvalue $\mu$. If $\lambda \neq \mu$, then $v$ and $w$ are orthogonal to each other.
$\blacktriangleright$ Proof.
We wish to show that $v \cdot w = 0$. We are given that $Av = \lambda v$, $Aw = \mu w$, and that $A = A^{T}$.
Note that $Aw - \mu w = \vec 0$, so we have $$
\begin{align*}
& v\cdot \vec 0 = 0 \\
\implies & v\cdot (Aw - \mu w) = 0 \\
\implies & v\cdot (Aw) -\mu (v\cdot w) = 0 \\
\implies & v^{T}Aw -\mu(v\cdot w ) = 0 \\
\implies & v^{T} A^{T} w - \mu(v\cdot w) = 0 \\
\implies & (Av)^{T} w -\mu(v\cdot w) = 0 \\
\implies & (\lambda v)^{T} w - \mu(v\cdot w) = 0 \\
\implies & \lambda (v\cdot w) - \mu (v \cdot w) = 0 \\
\implies & \underbrace{(\lambda - \mu)}_{\neq 0} (v\cdot w) = 0 \\
\implies & v\cdot w = 0
\end{align*}
$$as claimed! $\blacksquare$
With this in mind, when we have a real symmetric matrix, eigenvectors from different eigenspaces will automatically be orthogonal to each other. However, eigenvectors from the same eigenspace need not be orthogonal. So if we are building an orthonormal eigenbasis, we might need to use Gram-Schmidt on an eigenbasis for each eigenspace.
**Example.** Orthogonally diagonalize the matrix $$
A = \begin{pmatrix}1 & 1 & 1\\1 & 1 & 1\\1 & 1 & 1\end{pmatrix}.
$$That is, factorize $A = Q D Q^{T}$ such that $Q$ is an orthogonal matrix, and $D$ is diagonal.
$\blacktriangleright$ Before we start, a sanity check. This matrix $A$ is real and symmetric, so by real spectral theorem we know $A$ is orthogonally diagonalizable.
Now to start, we construct an eigentable for matrix $A$ as before.
The characteristic polynomial of $A$ is $$
p(t) = \det(A-t I ) = \det \begin{pmatrix}1-t & 1 & 1\\1 & 1-t & 1\\1 & 1 & 1-t\end{pmatrix} = -t^{2}(t-3),
$$so the eigenvalues are $\lambda = 0,0,3$.
Next we compute the eigenspaces $E_{0}$ and $E_{3}$, they are $$
E_{0} = \ker \begin{pmatrix}A-0I\end{pmatrix} = \ker \begin{pmatrix}1 & 1 & 1\\1 & 1 & 1\\1 & 1 & 1\end{pmatrix} =\cdots = \operatorname{span} \{\begin{pmatrix}-1\\1\\0\end{pmatrix}, \begin{pmatrix}-1\\0\\1\end{pmatrix}\}
$$and $$
E_{3} = \ker (A-3I) = \ker \begin{pmatrix}-2 & 1 & 1\\1 & -2 & 1\\1 & 1 & -2\end{pmatrix} = \cdots= \operatorname{span} \{\begin{pmatrix}1\\1\\1\end{pmatrix}\}.
$$So we have eigentable $$
\begin{array}{c|c|c}
\lambda & 0 & 3 \\ \hline
\text{am} & 2 & 1 \\ \hline
\text{basis of } E_{\lambda} & \begin{pmatrix}-1\\1\\0\end{pmatrix}, \begin{pmatrix}-1\\0\\1\end{pmatrix} & \begin{pmatrix}1\\1\\1\end{pmatrix} \\ \hline
\text{gm} & 2 & 1
\end{array}
$$
If all we want is to diagonalize the matrix $A$, we are done here, we have an eigenbasis to build an invertible $P$ and diagonal $D$ so that $A=PDP^{-1}$. However that's not what we want -- we want more. We want to orthogonally diagonalize $A$, that is, factorize $A$ as $A = QDQ^{T}$ where $Q$ is an orthogonal matrix (square with orthonormal columns), and $D$ diagonal.
Now by our lemma, we know the eigenvectors from $E_{0}$ will be orthogonal to eigenvectors of $E_{3}$, so we don't need to worry about the orthogonality between eigenspaces. However, within each eigenspace, we may need Gram-Schmidt to produce orthogonal eigenvectors. Indeed, for $E_{0}$, we have basis $$
w_{1} = \begin{pmatrix}-1\\1\\0\end{pmatrix} , w_{2} = \begin{pmatrix}-1\\0\\1\end{pmatrix}
$$which are not orthogonal yet, so we apply Gram-Schmidt on $w_{1}, w_{2}$, to get $$
\begin{align*}
u_{1} & = w_{1} = \begin{pmatrix}-1\\1\\0\end{pmatrix} \\
u_{2} & = w_{2} - \text{proj}_{u_{1}}w_{2} = \begin{pmatrix}-1\\0\\1\end{pmatrix} - \frac{1}{2} \begin{pmatrix}-1\\1\\0\end{pmatrix} =\begin{pmatrix}-\frac{1}{2}\\-\frac{1}{2}\\1\end{pmatrix} \parallel \begin{pmatrix}-1\\-1\\2\end{pmatrix}
\end{align*}
$$
So lastly, we can normalize this orthogonal eigenbasis,$$
\begin{array}{} \\
\text{from }E_{0} : \\
\begin{pmatrix}-1\\1\\0\end{pmatrix} \rightsquigarrow \frac{1}{\sqrt{2}}\begin{pmatrix}-1\\1\\0\end{pmatrix} \\
\begin{pmatrix}-1\\-1\\2\end{pmatrix} \rightsquigarrow \frac{1}{\sqrt{6}}\begin{pmatrix}-1\\-1\\2\end{pmatrix} \\
\text{from } E_{3} :\\
\begin{pmatrix}1\\1\\1\end{pmatrix} \rightsquigarrow \frac{1}{\sqrt{3}}\begin{pmatrix}1\\1\\1\end{pmatrix}
\end{array}
$$Hence, $A$ has orthogonal diagonalization $A = QDQ^{T}$ where $$
Q = \begin{pmatrix}-\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} \\
0 & \frac{2}{\sqrt{6}} & \frac{1}{\sqrt{3}}
\end{pmatrix} \text{ and } D = \begin{pmatrix}0 \\ & 0\\ & & 3\end{pmatrix}.\quad\blacklozenge
$$
**Remark.** Since here the eigenspace $E_{0}$ has dimension $\ge 2$, we needed to perform Gram-Schmidt on the basis for $E_{0}$ to make them orthogonal, and then orthonormal. If an eigenspace is only 1 dimensional, then we only need to normalize it. This is for real symmetric matrices only, of course.